Chernoff-Type Concentration of Empirical Probabilities in Relative Entropy

We study the relative entropy of empirical probability vector with respect to true in multinomial sampling k categories, which, when multiplied by sample size n, is also log-likelihood ratio statistic. generalize a recent result and show that moment generating function statistic bounded polynomial degree n on unit interval, uniformly over all vectors. characterize family polynomials indexed (k,n) obtain explicit formulae. Consequently, we develop Chernoff-type tail bounds, including closed-form version from large expansion bound minimizer. Our dominates classic method-of-types competitive state art. demonstrate an application estimating proportion unseen butterflies.